Multiple Plane Segmentation Using Optical Flow

In this paper we present a motion based segmentation algorithm to automatically detect multiple planes from sparse optical ¤ow information. An optimal estimate for planar motion in the presence of additive Gaussian noise is £rst proposed, including directional uncertainty of the measurements (thus coping with the aperture problem) and a multi-frame (n > 2) setting (adding overall robustness). In the presence of multiple planes in motion, the residuals of the motion estimation model are used in a clustering algorithm to segment the different planes. The image motion parameters are used to £nd an initial cluster of features be-longing to a surface, which is then grown towards the surface borders. Initialization is random and only robust statistics and continuity constraints are used. There is no need for using and tuning thresholds. Since the exact parametric planar ¤ow model is used, the algorithm is able to cope ef£ciently with projective distortions and 3D motion and structure can be directly estimated.